Just practicing

Django + SQLite + Collations

2009-03-18T11:25:00.002+03:00

There is rather well-known "SQLite case-insensitive problem", which is that SQLite can do case-insensitive SELECT only for ASCII character set. This problem was discussed at Django users list, posted as Djungo feature request and documented in the documentation:


  SQLite doesn't support case-insensitive matching for non-ASCII strings. Some 
  possible workarounds for this are `documented at sqlite.org`_, but they are 
  not utilised by the default SQLite backend in Django. Therefore, if you are 
  using the ``iexact`` lookup type in your queryset filters, be aware that it 
  will not work as expected for non-ASCII strings.

I also met this problem and I had to find a solution since I need to deploy one small application (ebooks catalogue) at the department machine where I didn't have root account and was unable to install another database engine.

The solution is not really hard.

Support of collations for SQLite is provided by loadable extension through ICU library, which is installed on almost every Linux box. This is documented here.

So first step is to get ICU extension for SQLite here and compile it with


  gcc -shared icu.c icu-config --ldflags -o icu.so

You need to have development files of libicu installed, for Debian/Ubuntu do


  sudo apt-get install libicu-dev

Now it is possible to test the extension (and also check if sqlite3 in your system is built with extension support):


  nazarov@heps1:~/lib> sqlite3
  SQLite version 3.6.11
  Enter ".help" for instructions
  Enter SQL statements terminated with a ";"
  sqlite> .load ./icu.so
  sqlite>  SELECT icu_load_collation('ru_RU', 'RUSSIAN');
  
  sqlite> SELECT * FROM core_author WHERE name LIKE 'вайн%';
  1420|Вайнер А.Л.
  4557|Вайнберг М.М.
  4558|Вайнберг М.М.
  6374|Вайнберг Б.
  7326|Вайнштейн Б.К.
  7327|Вайнштейн Л.А.
  7458|Вайнштейн С.И.
  8636|Вайнберг С.
  8958|Вайнштейн С.И.

If your SQLite is built without extension support, get appropriate version at SQLite homepage.

Then we need extension support in pysqlite. Since version 2.5.2 pysqlite can load SQLite extensions, but this feature is disabled by default. So download pysqlite, extract, remove the line


  define=SQLITE_OMIT_LOAD_EXTENSION

from the file setup.cfg and build it:


  wget http://oss.itsystementwicklung.de/download/pysqlite/2.5/2.5.5/pysqlite-2.5.5.tar.gz
  tar xzf pysqlite-2.5.5.tar.gz
  cd pysqlite-2.5.5
  vi setup.cfg
  python setup.py build

Now you can check, that pysqlite can load icu and make case insensitive selects:


  export PYTHONPATH=`pwd`:$PYTHONPATH;export LD_LIBRARY_PATH=`pwd`:$LD_LIBRARY_PATH;python
  Python 2.5 (r25:51908, Nov 27 2006, 19:14:46) 
  [GCC 4.1.2 20061115 (prerelease) (SUSE Linux)] on linux2
  Type "help", "copyright", "credits" or "license" for more information.
  >>> from pysqlite2 import dbapi2 as sqlite3
  >>> con = sqlite3.connect("lib.db")
  >>> con.enable_load_extension(True)
  >>> con.load_extension("./icu.so")
  >>> con.execute("SELECT icu_load_collation('ru_RU', 'RUSSIAN');")
  <pysqlite2.dbapi2.Cursor object at 0xb7cab660>
  >>> con.execute("SELECT * FROM core_author WHERE name LIKE 'вайн%';")
  <pysqlite2.dbapi2.Cursor object at 0xb7cab820>

Also I had to build static version of pysqlite which included SQLite, because SQLite on target system didn't support loadable extenstion. I did it with


  python setup.py build_static

So now we need to tell Django to use collations. Correct way would be to write custom db backend, but it is far simplier just to add 3 following lines to the file django/db/backends/sqlite3/base.py:


*** 143,148 ****
--- 143,153 ----
              }
              kwargs.update(self.options)
              self.connection = Database.connect(**kwargs)
+             # Add Russian collation
+             self.connection.enable_load_extension(True)
+             self.connection.load_extension("/home/nazarov/lib/icu.so")
+             self.connection.execute("SELECT icu_load_collation('ru_RU', 'RUSSIAN');")
+ 
              # Register extract, date_trunc, and regexp functions.
              self.connection.create_function("django_extract", 2, _sqlite_extract)
              self.connection.create_function("django_date_trunc", 2, _sqlite_date_trunc)

Now Django applications can do case-insensitive SELECTs (iexact, icontains etc) for Russian language.

On my target machine I had to start Django development server with


  export PYTHONPATH=`pwd`/Django-1.0.2-final/:`pwd`:$PYTHONPATH
  export LD_LIBRARY_PATH=`pwd`:$LD_LIBRARY_PATH;
  python library/manage.py runserver 10.0.0.15:8000

since I could not do system-wide install.

Popular history of particle accelerators experiments (part 1)

2008-06-30T09:27:00.003+04:00

Large Hadron Collider (LHC) will start operation this year, so there are a lot of fuss in press and blogs. Some people try to obtain public attention with legal claims against CERN. Nonetheless, general public is not aware of particle physics and accelerator experiments, and conceive LHC as final ultimate device for discovery of theory of everything or even as a doomsday machine. I want to review the history of particle physics experiments to show that LHC is next step in research. It shouldn’t be viewed as final or outstanding.

My posts are based on lectures of I.N. Meshkov at sixth winter school "Introduction to theory of fundamental interactions" in Dubna.

Ancient Greeks proposed two opposite ideas of structure of matter. One conception was that everything consists of several kinds of indivisible particles while the other told that everything could be divided into infinitely small parts.

So the target of particle physics is to find out which point of view is closer to real world and to study properties of particles for better understanding of our world which consists of particles.

Today we know that there are several kinds of elementary particles, which are united into so called “Standard model”.

Рис. 1:

Standard model of elementary particles

String theory tell us that all these particles are modes of oscillation of more elementary objects, which are called "strings but now it is just a hypothesis which could not be checked with experiment.

History of particle physics started at 1895 from discovery of photons as X-rays particles by Konrad Rentgen and discovery of electron by Joseph John Thomson (1897). First experiments didn’t use particle accelerators.

Рис. 2:

Konrad Rentgen

Рис. 3:

Joseph John Thomson

First electrostatic accelerators were created in 1931-1932 by R. Van-der-Graaf and J.D. Cockroft with E.T.S. Walton, and allowed the discovery of neutron by James Chadwick. He used reaction

These early accelerators were not very powerful and sophisticated, so many discoveries in particle physics were made from observations of cosmic rays, which started at 1912 by, Victor Hess. He has found in experiment with 3 electrometers carried out in a free balloon flight up to an altitude of 5300 meters that the ionization rate increased approximately fourfold over the rate at ground level. He concluded :"The results of my observation are best explained by the assumption that a radiation of very great penetrating power enters our atmosphere from above".

In 1932, Carl D. Anderson discovered positron by passing cosmic rays through a "Wilson chamber"with lead plate surrounded by a magnet to distinguish the particles charge. It was the beginning of the competition of accelerators with cosmic rays.

Cosmic rays consist of particles produced in outer space in thermonuclear reactions in stars and during supernova bursts. Some of these particles have energy which greatly exceeds energy of LHC and all the accelerators possible in near future. So it is one of the reasons why LHC wouldn’t destroy the Earth. But in spite of such enormous energy cosmic rays are rather inconvenient for particle physics experiments, because the density of such high-energy particles is very low.

In 1929-1932 Ernest O. Lawrence invented the resonance acceleration method which allowed construction of first cyclotron and first resonance linear accelerator, but next advance in particle physics was achieved in cosmic rays observations at 1936. Carl D. Anderson discovered of "mesotron which was supposed to be hypothetical Yukawa particle carrying interaction between protons and neutrons in nuclei. Later it was found out that mesotron was not Yukawa particle, but other electron-alike fundamental particle, so it was renamed to muon.

Break-through in accelerator technics was made by Vladimir I. Veksler (USSR, 1944) and Edwin McMillan (USA, 1945) with the discovery of the “Phase stability principle” and construction of the first electron synchrotron (Veksler) and the first synchrocyclotron (E.McMillan).

Рис. 4:

Vladimir I. Veksler

Рис. 5:

Edwin McMillan

Рис. 6:

Vladimir I. Veksler and Edwin McMillan at JINR (Dubna) 1958

Cosmic rays observations also made big progress, in 1947 pions, -hyperon and kaons were discovered. But cosmic rays observations were not very accurate, so there were some false “discoveries”, such as “Varitrons”. So first synchrocyclotrons allowed to improve results of cosmic rays observation.

In 1947 Berkeley Synchrocyclotron was put into operation. It had 184 inches (4.7 m) in diameter, and energy of accelerated -particles of 400 MeV. On this accelerator Lattes and E. Gardner achieved first artificial production of pions in reaction

Similar synchrocyclotron “Phasotron” was built in Dubna (USSR) at 1949. It is still in operation now and is used for cancer therapy, muon-catalysis, pion and muon physics experiments and nuclear physics experiments.

Рис. 7:

Phasotron, JINR (Dubna)

Since the end of 1940-th accelerators are the main tools of particle physics and majority of experimental results of fundamental character has been obtained both in particle, and nuclear physics at accelerators.

First day in our office

2008-03-04T01:26:00.001+03:00

Our project is getting real.
At Saturday we had the first day in our new office at the outskirts of Saint-Petersburg.
We have big room in the building of former headquarters of an aluminium plant.

Our first tasks were to set up LAN and internet connection, assemble office chairs and put up a whiteboard.

You can see that we successfully fulfilled these tasks!

Business-consumer social networks presentation

2008-01-18T22:01:00.000+03:00

Presentation byYury Lifshits on the project of creation of prototype of social network for interaction between businesses abd consumers.

| View | Upload your own

Project page

Half a year of absence

2008-01-07T21:32:00.000+03:00

I haven't written anything here for half a year.
It's a pity, but I had not much of enthusiasm for writing, and all of it I put into writing my master's thesis. So diploma is almost finished, and now I am preparing presentation.
Also I've presented my results at the local conference.
You can view my thesis and presentations here (sorry, in Russian only).
The main result is that we can conclude that QFT on non-commutative spaces is sufficiently non-trivial even in simplest case, studied (with opposite results) by Fiore and Wess.
Also I was able to construct several new non-commutative spaces.
Somehow I raised the entropy, because now we can study not one but several examples of non-commutative spaces, constructed with twisting :)

First bits of PyQt

2007-07-31T16:39:00.000+04:00

Yesterday I wrote my first PyQt application. It is really simple program, called "tray", which lives in system tray and notifies user of new messages at my favorite social network Vkontakte.ru (Russian clone of FaceBook.com).
The program is less than 170 lines of code. You can try it if you are registered at vkontakte.ru, but the program doesn't matter now.
I am very pleased with simplicity of creation of portable rich GUI applications with PyQt, but as always there are some issues on Windows. I created the program on my favorite old Linux box with vim, so there I had no problems with PyQt4 installation and no need of packaging, because the program consists of one file and three icons.
Then I decided to try it on Windows. First of all I installed newest PyQt 4.3.0. This new version is now packed in self-installing archive, which includes Qt and Eric4 IDE, so now you shouldn't have problems which I described in this post.
After that the program worked perfectly, but I needed to create standalone version for people who don't have Python and PyQt on their PCs.
This is rather simple task, but I've spent several hours fighting some issues.
I used py2exe to create standalone executable. The problem was to include PyQt into it. The idea is simple: I needed to specify that PyQt should be included. So I used solution from py2exe.org. I created file called setup.py with following contents:

from distutils.core import setup
import py2exe
setup(windows=[{"script":"tray.py"}], options={"py2exe":{"includes":["sip"]}})

Then I ran python setup.py install py2exe.
But I received errors like "module import failed: _qt". So I decided to mention inclusion explicitly: python setup.py install py2exe --includes=PyQt4,sip. It didn't help, but error message changed to "cannot find QApplication". Then I tried to reorganize imports in my code:

import sys
from PyQt4.Qt import *
from PyQt4 import QtCore, QtGui

gave correct result, and the executable was created.
I believe that this problem with imports is temporary and will be fixed in near future.

So then I used Inno Setup to create setup package and it was really easy.

Conclusion
It is really easy to create portable applications with great GUI with PyQt4.
The only trade off is the size of installed application on Windows: for my 170 lines of code project I've got 5 Mb setup package, which extracts into 17 Mb application.
The reason is simple - lack of common packaging system with dependencies on Windows, so the program needs it's own copy of Python and PyQt.

My PCs

2007-07-13T13:01:00.000+04:00

Today I will try to retrospect history of my home PCs. It is rather personal, so please leave the page immediately if you are not interested :)

For those, who haven't left yet.
My father is a software engineer, so there is nothing unusual that we'd got first home computer as soon as was possible. It was in 1990 or 1991, when the Soviet Union still existed, so there were some problems with foreign devices. Our first computer was hand-made ZX Spectrum clone, which was assembled from different foreign and Russian chips by my father's friend. It had rather peculiar hand-made keyboard, and used Kometa-212 tape-recorder for data storage:
,
and monochrome TV Kaskad-203 as the display, which looked like this:

I'd just started to play chess in local center for youth creativity and enjoyed playing with this computer very much. I was unable to beat it at that time, since I was only six or seven years old :)
Also I tried to do some programming in built-in Basic, but without much success, since I was unable to save entered program on the tape.
I will try to find and photograph remains of that computer.

It's a pity that couple of years after our old TV ceased to function, so we bought another TV, which was incompatible with computer output.

So for couple of years we had no working computer at home. I should mention, that the beginning of 1990s was rather hard time in Russia, so IBM PCs were to expensive for us. They cost more than $1000, and it was very big money. The average wage at that time was about 50-100 USD a month.

When I was eleven, I started study programming in Palace of Youth Creation. I was taught Turbo Pascal 7.0. So I was really happy when we bought a computer as a New Year gift at the end of 1996.

It was rather good and expensive machine with AMD K5 75 MHz processor, 8 megabytes of RAM, 600 megabytes hard drive and brand new Windows 95 OS and 14" CRT monitor GoldStar 1468.

I still use this monitor, which has rather decent picture quality and 800x600/75Hz resolution.

We also bought a printer HP DeskJet 400, which is really cool, I used it last time just two years ago, when I had no time to buy a cartridge for new printer.

I enjoyed playing Warcraft II and programming in Turbo Pascal. But the OS was a complete disaster, it showed Blue Screen of Death several times a day. May be it was due to the lack of system resources, since OS became more stable when we added 8 more megabytes of RAM a year later, or may be because the system was cracked, not bought legally. As you remember, couple hundreds of bucks were a lot of money in Russia in that time :)

In the end of 1997 we added 8 megs of RAM, sound card, CD-ROM and 1.2 Gb hard drive to our PC. Nevertheless, by the end of 1998 it became completely obsolete.
So we "upgraded" it by replacement of main board, processor, RAM, and hard drive. We used the box of old PC, and also bought new LG Flatron 795FT monitor.

New computer was pretty cool, it was based on AMD K6-2/350Mhz processor, with 64 megs of RAM (128 more megabytes was added a year later), 6 Gb Seagate Medalist hard drive and great USR Courier 56k modem. Modem was really good on our old phone line. We had almost no breaks, but rather slow (33,6k) connection because of old hardware at local exchange.

In 1999 computer section in Palace of Youth Creation switched from DOS/Windows 3.11 to FreeBSD, so I started to study C, C++ and Perl, and installed RedHat Linux 5.2 and later 6.0 and 6.2 on our first hard drive, which was 600 megabytes. It's funny that there were enough space for X11, KDE, XEmacs, LaTeX, kernel sources and so on. I also enjoyed Fallout 2 game.

In the end of the year 2000 we decided that we need new PC, so we bought completely new AMD Athlon XP 1733+ MHz with 256 megs of RAM and 40 Gigs IBM hard drive. We plugged newer LG monitor into new PC, and old GoldStar to older one. So for the first time we had more than one working computer :)

Well, later we added new hard drives to new PC, 512 megabytes of RAM, CD-RW/DVD-RW drives, new Canon PIXMA iP4000 printer, Epson scanner and so on, but that is not very interesting. In the beginning of this year 40 Gb IBM hard drive on older PC failed, so that computer became Linux only box. I installed KUbuntu 7.04 on it and I use it now everyday for software development, and it is rather fast :) It can host Apache, MySQL and PHP, also I do C++ and Python development on it. It even can play Youtube movies :) The only thing that I've done for optimization was the replacement of KDE with Fluxbox.

QFT on non-commutative spaces

2007-07-05T11:23:00.000+04:00

Fiore and Wess in their article ”On full twisted Poincare’ symmetry andQFT on Moyal-Weyl spaces” shows that non-commutative QFT withproperly enforced ”twisted symmetry” is equivalent to ordinary commutativeQFT. They start from enforcing space non-commutativity in most simleway:

(1)

Where is a constant matrix. So the limit gives ordinarycommutative space. Then algebra generated by is equivalent to algebra offunctions on commutative space with deformed product, called star-product.Such star product is introduced using twist :

(2)

Properly chosen gives

(3)

as required.

There are whole classes of equivalent twists, which lead to the same starproduct, so the theory doesn’t depend on the choice of particular twist. ThenFiore and Wess show how to construct QFT starting with . To obtain properself-consistent theory they have to change Poincare-invariance with invariancewith respect to deformed Poincare group, obtained from deformation of Poincarealgebra with the same twist , as was used to deform algebra of functions.

After the introduction of Wightman axioms and study of Wightman andGreen’s functions it turns out, that this functions coincide with their undeformedcounterparts at least perturbatively. So deformed Poincare symmetry works as”compensation” of space non-commutativity, and theory gives no newphysics.

But there are another possibilities. We can use more complex non-commutativespace, for example, we can start from deformation of Poincare symmetry to-Poincare, then introduce a twist on -Poincare and build non-commutativespace with twisted -Poincare symmetry.

I hope to study this case in details in the next several days.

Creating simple yet powerful anonymous downloader in Python

2007-06-28T16:59:00.000+04:00

Let's suppose for example that we have a task to download big number of pages from some web server. It may be something all personal pages of server users, list of postal addresses or firms.
Python is a very convenient language for this task because it has Interactive Mode (or REPL - Read-Eval-Print-Loop). So we can start Python and write something like this:

>>>import urllib,re # some modules which we will need later

If target web site has some kind of index page, we should start from getting it and parsing it into items:

>>>index_page=urllib.urlopen('http://example.com/index').read() # we have read contents of the page
>>>items_to_get=re.findall('some regexp',index_page) # now we have list of all items to download, regexp may be something like '<a href=\'([^']*)\'>'
>>>out=open('index','w') # let's save all the items for future use
>>>for item in items_to_get:
... out.write(item+'\n')

>>>out.close()

It is really simple!
Now we can get all the pages. Let's suppose, that items are simple pages, something like 001.html

>>>for item in items_to_get:
... open(item,'w').write(urllib.urlopen('http://example.com/%s'%item).read())
...

This two lines of code will download and save all the pages into files like 001.html

But there are two big problems. First, we download only one page a time, so it will take a lot of time to get all the pages even in case of broad connection because of delays on sending requests, writing pages to disk and so on. Second and more important: site admin can notice that someone (you) tries to download the whole site and ban your IP.

So we need to write real piece of code.
I will write it in separate file getter.py:


 1 #Let's import something
 2 import os,urllib,urllib2,socket,sys,re,threading,time,random
 3 
 4 max_thread_num=10 # we will download in ten threads!
 5 
 6 
 7 # Let's create class for a thread of download
 8 class ProxyThread(threading.Thread):
 9         def set_params(self,proxy): # proxy address
10                 self.proxy=proxy
11 
12         def run(self): # main method of thread
13                         # first we create proxy_handler to handle connection with anonymous http proxy
14                         proxy_handler = urllib2.ProxyHandler(self.proxy)
15                         auth = urllib2.HTTPBasicAuthHandler()
16 
17                         # then we build opener, wich works through the proxy
18                         opener = urllib2.build_opener(proxy_handler, auth, urllib2.HTTPHandler)
19                         #some fake User-Agent (may be useful for some sites)
20                         opener.addheaders = [('User-agent', 'Mozilla/5.0')]
21                         #Let's now check connection with proxy by getting some data from google
22                         try:
23                                 urllib2.urlopen('http://google.com').read(5)
24                         except:
25                                 # this proxy doesn't work, we will try another. 
26                                 #We need to increment number of available threads. 
27                                 #Simply use global variable
28                                 global max_thread_num
29                                 max_thread_num+=1
30                                 return
31 
32                         # this proxy is good, so we get items, wich are needed to be downloaded
33                         items_to_get=open('index').readlines()
34                         # we shuffle list so that different threads try to download different items
35                         random.shuffle(items_to_get)
36 
37                         for itemstr in items_to_get:
38                                 item=itemstr.rstrip() #strip newline symbol
39                                 if os.path.exists(item): #Already downloaded ?
40                                         continue
41 
42                                 #Let's try to download something :)
43                                 try:
44                                         data=opener.open('http://example.com/%s'%item).read()
45                                         #CoDeeN proxies give greetings page on first request, so we need to skip it
46                                         if not re.search('CoDeeN',data):
47                                                 open(item,'w').write(data)
48                                 except:
49                                         #If there was some error during download, 
50                                         #we will remove incompletely downloaded item, 
51                                         #and skip this proxy, because majority of errors are proxy errors:
52                                         try:
53                                                 os.remove(item)
54                                         except:
55                                                 pass
56 
57                                         global max_thread_num
58                                         max_thread_num+=1
59                                         return
60 
61                         global max_thread_num
62                         max_thread_num+=1
63                         return
64 
65 # Well, it was quite a long method mostly because of error handling. 
66 
67 #Now small piece of code wich gets list of proxies and starts threads:
68 socket.setdefaulttimeout(120) # increase connection timeout because some proxies respond after rather long time
69 #We will go trough all 50 pages of free proxy list at http://www.samair.ru
70 for num in range(50):
71         # We get page of proxies
72         proxy_page=urllib.urlopen('http://www.samair.ru/proxy/time-%02d.htm'%num).read()
73         # and search for all proxy addresses on it
74         proxies=re.findall('<span class="proxy\d*">(\d+)</span>.<span class="proxy\d*">(?P<n2>\d+)</span>.(\d+).(\d+):\s*(\d+)',proxy_page)
75         # rather complex regexp :)
76         for proxy in proxies: # try to start thread for each proxy
77                 while max_thread_num<=0: # all 10 thread have started, so we need to wait 
78                         time.sleep(1)
79                 # Then we can start a thread: 
80                 prox={'http':'%s.%s.%s.%s:%s'%proxy}
81                 pt=ProxyThread()
82                 pt.set_params(prox)
83                 max_thread_num-=1
84                 pt.start()

Then we can start the program python getter.py.
Well, it works!

Almost :)
I've written no ending condition, so this code will end not when all the items will be downloaded, but when all the proxies will be checked. Anyway you need to decide which items you should download and what to do in case of errors.

Well, it looks like the next post can be about spam bots or other kinds of malware :)
Nonetheless, I believe that you won't use the code for illegal actions :)

Fit2PDA moved to Google Code, new project started

2007-06-22T17:17:00.000+04:00

Today I moved Fit2PDA development to Google Code. Thanks to geomatsi for proposal!

All the source code is in svn now. Also I've written some help pages for project and uploaded build for Windows.
Linux build will be ready tomorrow.

So please feel free to download try and write bug reports

I will be happy to get new ideas and patches too!

New project vkontakte.net.ru

Last two weeks there was very little activity on Fit2PDA from my side, because of my new web project vkontakte.net.ru.
It is devoted to research on most successful Russian social network vkontakte.ru. I created small program in Python to draw maps of connections between people in this network. Web site has wiki for discussion of new ideas.

I need to warn you that vkontakte.net.ru is and will be in Russian only. Anyway why one not reading Russian can be interested in research of Russian social network :)

You can visit vkontakte.net.ru for more information if you read Russian.

Eric IDE on Unfriendly System

2007-06-04T16:17:00.000+04:00

At morning I've read on linux.org.ru of new version of Eric IDE. After looking at screenshots I was very interested in this IDE.

So I decided to install it on my work PC.
Unfortunately it's running Windows XP, and I can't change it. And those OS doesn't have normal package manager.

To install Eric I needed Qt, Python, PyQt, QScintilla.

I had opensource Qt (installed with MinGW) and Active Python installed on my PC.
So I downloaded PyQt, QScintilla and Eric.
I installed binary package of PyQt and started to build QScintilla with MinGW. The build was successful, but python bindings didn't installed because of some error with SIP. So I tried to build SIP separately. It wasn't build because of some unknown linking errors with Python libraries.
I searched trough the internet, but was unable to find solution.
So I decided to reinstall everything.

First of all, I downloaded Python from python.org and installed it (it was what really helped).
Then I reinstalled Qt.
Then I downloaded source package of PyQt, unpacked it and run

python configure.py

make

make install

The build completed with no errors.
Next I downloaded QScintilla source and unpacked it. Then I went to Qt4 subdirectory, and run

qmake qscintilla.pro

make

make install

After that I went to Python subfolder of QScintilla source, and ran

python configure.py

make

make install

When the build was completed I had all needed to run Eric4.
(Also SIP is included into PyQt, so one don't need to build it separately).
Then I unpacked Eric and run
python install.py
eric4.bat
It started after all!

Eric is really great, but I've met the problem on second run - it haven't started because of some error in debugger configuration. So I had to turn remote debugging and passive debugging on through eric-configure.bat. After it the IDE started normally.

Time to upgrade Ubuntu on my home boxes

2007-05-26T18:17:00.000+04:00

Today I've decided that the time to upgrade Ubuntu Linux on my home PCs has come. Yesterday I received Ubuntu 7.04 and Kubuntu 7.04 CDs from Canonical.

I have two PCs at home: old AMD K6-2/350 MHz with 40 Gb dying primary hard disk (Windows 2000 is installed on it :)) and 6 Gb secondary drive, where Ubuntu 5.10 was installed, and newer Athlon XP 1800 with Ubuntu 6.10 on it.

It's a pity that you can not use live/install Ubuntu cd for upgrades :(

So I've decided to do dist-upgrade on newer PC and completely reinstall system on older. I've chosen Kubuntu for old box, because it includes Konqueror, which is more lightweight than Firefox.

Several hours ago I've back-upped all important data from my older pc and started installation of Kubuntu. I've already met several problems:

I need to write acpi=force in the boot options of live-cd, because BIOS was released before the year 2000.

Installer was unable to format partition into XFS because of some unknown error (possibly connected with dying primary drive), so I had to create partitions by hand using QParted.

May be I'd better to write hda=noprobe boot option to prevent errors from accessing dying primary drive, but I haven't done it

Now the installation process is 94% complete.

And almost half of required packages have been downloaded on my newer PC through my slow 200 kbit ADSL connection.
So keep on touch with me, I will report the progress and problems!

Update 1:
The installation of Kubuntu is finished, now it is time to test the system and install additional packages.
I plan to install:

LaTeX

maxima

developer's tools

java

Mp3 and dvd playback

Open SSH

Update 2:
Installation of mp3 playback was really easy - I tried to play mp3 file and Amarok asked me to install mp3 support, I answered yes and in couple of minutes it started to play.
There were no problems with OpenSSH server too: sudo apt-get install openssh-server
But Adept apt front-end is really slow :(
Well, I installed almost everything I wanted, only couple of packages left.
And download of packages for upgrade on newer PC has not finished yet.

Update 3(next day):
All updates and installations are finished, and now I'm happy with two up-to-date systems.
Rhythmbox on my newer PC now plays last.fm. I'm also happy with kopete LaTeX plugin

On educational system

2007-05-23T11:44:00.000+04:00

I've just read this blog entry about crisis in British higher education, created by lowering of standards and payments and introduction of modularity and lecturers audits.

The author of this article works in area of Computer Science, but the tendency seems to be universal both in UK and US. (You can look, for example, through this post)

So why we in Russia should be interested in problems of far richer countries?
Our educational system is in decline because of enormous and long-lasting lack of money. But modern reformation efforts introduce new dangers.

Today we are accepting participation in so called "Bologna process", which will inject to our education some problems of British system, mentioned in the first article. The most obvious one is modularity, which allows students to choose courses to learn. It leads to lack of interest in harder core courses, and I can feel it myself. Today I have passed in two of four courses which were given in our department (of QFT), and I have no need to pass the tests on other two, which are far harder.

Nevertheless, "Bologna process" is a minor threat comparing with government educational policy.

By the way, our problems will inflict western education too, since it is supported by immigration from our country.

The only hope is academic emigration from China and other developing countries.

Draft of the article

2007-05-11T11:15:00.000+04:00

I've written an article on reading on PDA. It gives some ideas how to read books, articles from arxiv and so on.
This is only first draft, so I will be happy to get your opinions and corrections, especially on my poor English.

This articles is intended to an advertisement of my project Fit2PDA :)

Fit2PDA

2007-05-09T21:10:00.000+04:00

I've finally chosen the name for my program.
So it is now called Fit2PDA.

I think that the name is apt to the program :)

I've done some additional improvements in GUI, so now you can see (rather accurate:) progress bars during conversion and you can convert not only PDF/PostScript, but also Djvu.

Big number of free scanned ebooks can be found in Kolhoz library and eDonkey2000 p2p-network.

By the way, there are 2 good compatible clients for eDonkey: eMule for Windows and aMule for Linux. So, if you use both operation systems, you can continue your emule downloads in amule. You just need to select the same working directory!

The book "Innovator's Dilemma"

2007-05-09T20:25:00.000+04:00

Couple of days ago I've finished the book "Innovator's Dilemma" by Clayton M. Christensen.
I read several times that it is "the best business book ever written" an so on.
Well, I don't know is it true, I haven't read a lot of business books.
The book is really interesting, but the main idea of the book, that there are so called "disruptive technologies", which can't be handled by existing businesses seems rather natural.
Why else new businesses would appear after all?

I was able to download this book for free from Internet shop bizbook.ru, because they give 20 pages preview of the book for free (in very poor quality images).
So I've used number of anonymous proxies and downloaded the whole book.
I had to do it because I was unable to find the book in usual warez sources and p2p networks.
It a pity that I had to do such a thing myself, because it means that we don't have access to all the books worth to read even through pirate's networks.

By the way, I used my program Fit2PDA to convert this book, and it worked perfectly :)

First "public" version of "qtsplitter"

2007-05-02T21:23:00.000+04:00

Finally first public alpha is out!
First completely working build for Windows is here
Simply download and extract archive, then run converter3.exe , choose pdf-files to convert, select correct screen width of your PDA and convenient resolution and press on button with red point. Then take a cup of coffee and wait while your papers or books are converted.
You will get iSilo-friendly html files, which could be easily converted to Palm-powered with iSiloX, or viewed with Pocket IE on PocketPC.
So I will be happy with any feedback from you!

Also I will be happy to get help with UI design and Linux package creation.

And the final question: do you think that "FitToPDA" is a good name?

New version of "qtsplitter"

2007-04-17T21:54:00.000+04:00

As you already know, I'm writing utility program for conversion of scanned scientific books to PDA and other devices with small screen size. There is a great need in such a program since there is no reliable and free formula recognition product.

I discovered quite simple solution - we don't need to recognize text, we can just reformat the images of pages to feet to PDA screen.

So that's all about history and motivation. Now let's talk about new version. In this version I incorporated deskew algorithm which automatically rotates image to make text horizontal. Also I reworked splitting code, and now the program is able to split colorful images (though, it uses only black-and-white data to find lines of text) and it tries to preserve under- and over- lines.
I was thinking a lot about coding style when writing this code, because I didn't have sufficient C/C++ practice last couple of years. So I will be happy to hear your suggestions on that matter.

Also I want to give sounding name to this program, so I declare some kind of contest for the best name, and I'm waiting for your proposals.

And one announcement: next version will have a GUI, so I plan to make it public. I will be happy with any kind of support.

Update:
I forgot to mention sexy example

Just finished reading the book

2007-04-12T21:40:00.000+04:00

Couple of days ago I finished reading the book "Advanced Linux Programming".
It is old but excellent book on programming in GNU/Linux environment. It starts from brief description of everyday developer's tools such as emacs, gcc, gdb, make and then leads to core topics of Linux development. It covers main system calls, IPC, networking, usage of shared libraries and plugins, error handling and so on.
So it is really useful for experienced developers and even for beginners because the text is very clear. There is no information on GUI development and minor libraries, so the book is very useful even six years after the publication.
It can be downloaded from the official site

I was doing some marks when reading, so I created some kind of reference card page on some covered in the book. You can find it here

About my projects

2007-04-10T20:55:00.000+04:00

I believe that you have already discovered that my blog is located on my home site.
I use it not only as photo album, but I also put here information related to my current free-time projects.

Now I try to make my qtsplitter program usable. Now it can be used as command line tool under both Windows and Linux.
This program splits pages of scanned books into lines and split lines into parts so that they fit into PDA display. You can see how it works here .
You can try the program yourself if you are really interested and know how to build Qt projects.
I would really appreciate if somebody could make and upload statically linked version for Windows.

Also I started to collect useful tips on Vim editor here. After all my site is really apt to such kind of projects because it is wiki-based.

The Beginning

2007-04-09T17:02:00.000+04:00

Hello World!